Papers with downstream dialogue tasks
Lightweight Transformers for Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Commercial dialogue systems typically require a small footprint and fast execution time, but recent trends are in the other direction, resulting in difficulties in model deployment. |
| Approach: | They build Transformer-based Language Models from scratch on large corpora of conversational data and compare their performance against BERT and other strong baselines on dialogue probing tasks. |
| Outcome: | The proposed model outperforms existing models on dialogue probing tasks and can be fine-tuned on a single consumer GPU card. |
DivTOD: Unleashing the Power of LLMs for Diversifying Task-Oriented Dialogue Representations (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing language models pre-trained on general text overlook the one-to-many property of task-oriented dialogues, where multiple responses can be appropriate given the same context. |
| Approach: | They propose a model that pre-trains LLMs to learn diverse task-oriented dialogue representations by removing domain knowledge that contradicts TODs. |
| Outcome: | The proposed model outperforms strong TOD baselines on various downstream dialogue tasks and learns the intrinsic diversity of task-oriented dialogues. |
Learning Dialogue Representations from Consecutive Utterances (2022.naacl-main)
Copied to clipboard
| Challenge: | Dialogue Sentence Embedding (DSE) is a self-supervised contrastive learning method that learns effective dialogue representations suitable for a wide range of dialogue-oriented tasks. |
| Approach: | They propose a self-supervised contrastive learning method that learns dialogue representations suitable for a wide range of dialogue tasks. |
| Outcome: | The proposed method outperforms baselines on five dialogue tasks on a few-shot and zero-shot datasets. |
Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to make multilingual systems expensive and tedious introduce pipeline of errors. |
| Approach: | They propose to use pre-trained multilingual models to enhance the transfer learning process by intermediate fine-tuning of pretrained multi-lingual models. |
| Outcome: | The proposed approach improves on the cross-lingual dialogue state tracking task with only 10% of the target language task data and zero-shot setup respectively. |
BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models lack diversity and linguistic challenges in task-oriented dialogues. |
| Approach: | They propose a self-bootstrapping dialogue pre-training model called BootTOD . it learns task-oriented dialogue representations via a framework . |
| Outcome: | The proposed model outperforms strong TOD baselines on diverse dialogue tasks. |
FutureTOD: Teaching Future Knowledge to Pre-trained Language Model for Task-Oriented Dialogue (2023.acl-long)
Copied to clipboard
| Challenge: | Existing pre-trained language models rely on a contrastive framework and are difficult to use in practice. |
| Approach: | They propose a dialogue pre-training model which distills future knowledge to the representation of the previous dialogue context using a self-training framework. |
| Outcome: | The proposed model can be applied to various downstream dialogue tasks. |